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ABSTRACT 

In Learning Networks, learners need to share knowledge with others to build knowledge. In particular, when working on 
complex tasks, they often need to acquire extra cognitive resources from others to process a high task load. However, 
without support high task load and organizing knowledge sharing themselves might easily overload learners’ limited 
cognitive capacities because learners first have to find relevant peer tutors (i.e., those who provide help) and then 
maintain the social interaction. We propose to design a peer-support system that selects tutors and provides support 
during knowledge sharing. The pilot study reported here investigated the effects of two peer tutor competences, tutoring 
skills vs. content knowledge, on tutees’ (i.e., those who need help) cognitive load and learning performance. The results 
show that tutees supported by tutors with tutoring skills experienced lower cognitive load and had better learning 
performance than did tutees supported by tutors with content knowledge. This is in line with our assumption, but for 
confirmation we need to gather more data in a full study. We need to first use a task that requires learners to rely on 
others to trigger higher cognitive skills to deal with high task load. Secondly we need to find a modus to ensure that the 
tutors follow the instructions to apply the particular competence. 
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1. INTRODUCTION 

A Learning Network (LN) is a particular kind of online social network that is dedicated to learning (Sloep 
2009). In LNs, learners are themselves responsible for sharing knowledge with their peer learners. When 
organizing knowledge sharing themselves, learners first have to find out relevant knowledge sharers and then 
maintain the social interaction with others to reach a shared understanding and knowledge building. To help 
finding relevant knowledge sharers, our colleagues have developed two tutor selection systems that 
automatically assign peer learners with relevant content knowledge to act as peer tutors to help answer each 
others’ content-related questions (Van Rosmalen 2008; De Bakker 2010). However, due to the heterogeneous 
group composition of LNs it is not always possible to find tutors with content knowledge related to tutees’ 
questions. In addition, the questions or problems that participants of LNs have are often rather complex: they 
are authentic problems originating from real-life contexts, such as the working place. When working on 
complex tasks, learners have to allocate many of their cognitive resources to process numerous information 
elements and element interactivity and this imposes a high cognitive load (Sweller et al. 1998). Cognitive 
load (or mental workload) refers to the learner’s limited cognitive capacity actually allocated on performing a 
particular task and it has been recognized as an important factor that influences learner performance (Sweller 
et al. 1998; Hart 2006). We surmise that tutors who provide content-related knowledge only cannot alleviate 
their tutees’ cognitive load. 

Furthermore, maintaining the social interaction to reach a shared understanding or build knowledge on 
complex tasks requires certain pedagogical and process -facilitation skills (King 2007; Roscoe and Chi 2008). 
These skills go beyond the knowledge transfer or information exchange that content tutors often offer. If it is 
not possible to always find tutors with content knowledge precisely related to tutees’ questions, nor do tutors 
with content knowledge guarantee effective tutee learning, it is therefore necessary to consider tutors with 
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tutoring skills. However, while most tutoring studies have examined effects of tutors’ content knowledge on 
diverse dependent variables of tutees, there have not been many studies that measured effects of tutors’ 
tutoring skills on tutees’ learning. Studies of reciprocal peer tutoring have compared the effects of supporting 
learners with and without tutoring skills on learner performance (King 1994; King et al. 1998; Nath and Ross 
2001). Their support of tutoring skills was to make sure that learners would demonstrate tutoring skills to 
elicit certain social interactions and trigger cognitive processes that contribute to learning. Some studies have 
shown that the learners with tutoring skills outperformed the control groups on knowledge or comprehension 
tests (King 1994; King et al. 1998). However, in LNs knowledge sharing involving peer support is more akin 
to asymmetric rather than reciprocal peer tutoring. From these studies we can therefore only speculatively 
claim that tutoring skills are likely to promote tutee learning. Considering human limited cognitive capacities, 
we need to redesign our previous tutor selection system to figure out what makes an effective tutor and to 
design support for the social interaction process. To achieve this, we first of all need to investigate the effects 
of peer-tutor competences, i.e. content knowledge vs. tutoring skills, on tutee learning. In this pilot study we 
investigate whether it is possible to determine an effect of the two types of peer-tutor competence when 
tutors do not receive prior training but instead receive a tutoring guide to follow. 


2. METHOD 

Two computer science classes jointly consisting of 28 students from a pre-university secondary school in the 
Netherlands took part in this pilot. We assigned students in half of classes to act as tutors and the other half as 
tutees. The tutor role was split; a tutor was either a tutoring skills (TS) or a content knowledge (CK) tutor. In 
total there were seven TS and seven CK pairs. The intervention of this study was to provide tutors with a 
tutoring guide including additional course materials (for CK tutors) or instructions of tutoring skills (for TS 
tutors) to assure that tutors had either CK or TS competence. The instructions that TS tutors received 
consisted of general rules for pedagogical and process-facilitation skills as well as specific step-by-step 
directions that guide tutors to perform pedagogical and process -facilitation skills (King 2007). This study 
consisted of two sessions in 2.5 hours. Two websites were set up for these two sessions: the course site 
contains the learning material of Sex and Evolution as well as the task site contains task instructions, chat and 
wiki tools. During the first session, students studied the learning material on the course site and then they 
took the pre-test and filled in the Tutoring Skills Questionnaire. There was a 15-minute break between these 
two sessions. The second session was started with a brief introduction of peer tutoring, task requirements and 
use of chat and wiki tools. Then students had 50 minutes to work in pairs to complete the essay task. While 
performing the task, students no longer could consult the course material. When working on the task, 
students had to use a chat to communicate with each other and they were not allowed to talk face-to-face. 
Tutees had to write the essay in a wiki and to “publish” the wiki page frequently to allow the tutors to read up 
on the tutee’ s progress. Tutors could only read the wiki page, but they could not edit it. To view the updated 
wiki page, tutors had to “refresh” the wiki page frequently. Tutors used a respective tutoring guide to help 
tutees work on essay answers through the chat. After the task, students had 20 minutes to indicate 
experienced cognitive load on the NASA-Task Load Index (see Table 1) (Hart 2006), to take the post-test 
and to fill in an evaluation survey. 


3. FINDINGS 

Pre-measures. Both groups of tutees estimated their tutoring skills to be quite high, 63 out of possible 75. 
The low pre-test scores showed that tutees of both groups did not have sufficient prior knowledge though TS 
tutees scored slightly higher than CK tutees. For both pre-measures, there was no difference between tutees 
assigned to TS or CK pairs. 

Tutee learning performance. Tutees' low post-test scores indicated that working on the essay task did 
not help students leant much related to the course materials. In addition to the same items of the pre-test, 
there were five task-related items in the post-test. CK tutees performed better on five task-related items than 
TS tutees: this means that additional course materials might help tutees understand the topics included in the 
essay task better than instructions of tutoring skills. Both groups of tutees performed equally well on essay 
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answers though TS tutees performed slightly better than CK tutees. Qualitatively, TS tutees answered more 
topics completely than CK tutees whereas CK-tutees answered more topics partially than TS-tutees. 

Tutee cognitive load. TS tutees’ total cognitive load is lower than CK tutees’ and lower on all six 
dimensions. In particular, TS tutees experienced much lower cognitive load on physical demand and 
frustration than CK tutees. 

Table 1. Tutee cognitive load on six dimensions of NASA-TLX 




Mental 

demand 

Physical 

demand 

Temporal 

demand 

Performance 

Effort 

Frustration 

Total 

TC 

Median 

12.00 

3.00 

10.00 

6.00 

9.00 

2.00 

41.00 

1 N 

SD 

4.27 

3.82 

6.05 

3.70 

4.08 

5.68 

14.60 


Median 

14.00 

11.00 

10.50 

8.00 

13.00 

10.50 

68.00 

v K 

SD 

5.59 

4.32 

4.66 

4.67 

3.93 

2.78 

20.01 


Tutee cognitive load and learning performance when their tutors applied the tutoring guide. Chats 
were analyzed for evidence of adherence to the assigned tutoring competence. Only two of the TS tutors 
applied the specific step-by-step instructions and five of the CK tutors referred to the additional course 
material. Table 2 shows that TS tutees performed better on all performance measures and they experienced 
lower cognitive load than CK tutees. 

Table 2. Performance measures and cognitive load of TS tutees and CK tutees supported by tutors who actually applied 

intervention during chats 


TS-tutees (n = 2) CK-tutees (n = 5) 



M 

SD 

M 

SD 

pre-test 

5.00 

2.83 

3.75 

.96 

post-test 

6.50 

3.54 

4.40 

1.52 

5 task-related items 

4.00 

0 

3.60 

1.67 

essay 

7.92 

1.29 

6.71 

1.78 

total cognitive load 

51.00 

14.14 

68.20 

10.85 


4. CONCLUSION AND DISCUSSION 

Although TS tutees had higher scores on the essay task and post-test, CK tutees seemed to have higher scores 
on the five task-related items than TS tutees. From how multiple topics were answered in the essays, we 
surmise that the different tutor competences influenced how tutees worked on the task. The diverse examples 
included in the additional course materials helped CK tutees answer more topics partially while the 
instructions that TS -tutors used helped TS tutees answer topics more completely. When looking into the data 
of tutees supported by tutors who indeed applied the intervention of additional course materials and tutoring 
skills, the results in Table 2 turned out to be different from the group data: TS tutees performed better than 
CK tutees on both test and task measures. During knowledge sharing, CK tutees in general experienced more 
cognitive load than TS tutees. In particular, the much lower score on frustration corresponds to findings of 
our previous study (Hsiao et al. in press). We observed that some CK pairs completed the task much earlier 
than the expected time of 50 minutes. Although using additional course materials helped CK tutees complete 
the task earlier, CK-tutees still indicated a high cognitive load. 

The findings of this pilot are hopeful in the sense that discerning CK and TS tutoring competences seems 
to make sense. In order to corroborate our first findings, we need to improve three aspects for the full study. 
First, in this study we tried out different performance measures in order to find out which could clearly 
answer the research question in the full study. The pre- and post-test that measured students’ understanding 
of the entire course could not appropriately gauge the effects of the intervention on the essay task since this 
essay only covered several topics of the course. Thus, to better measure learning performance, we should 
either expand the number of task-related items to a longer test or use the essay answers that directly reflect 
how students learn from working on the task. Second, tasks with different complexities require different 
amount of cognitive resources to deal with the task load. Complexities of essay questions not only depend on 
the interactivity of information elements but also on the level of cognitive skills required. The essay question 


267 



ISBN: 978-989-8533-12-8 © 2012 IADIS 


of this study is still a relatively simple task: comparing and contrasting facts require a lower level of cognitive 
skills, namely understanding. From satisfactory essay scores, we surmise that tutees might be able to answer 
the essay by merely retrieving learned factual knowledge from their own or their tutors’ memory instead of 
performing higher cognitive skills to process the task load. This corresponds to our model that when working 
on simple tasks learners do not need to resort to knowledge sharing or that knowledge sharing does not 
contribute to better learning effects (Hsiao et al. 2011). Third, considering the online characteristics of LNs, 
we supported tutors with certain competences by giving them ready-to-use tutoring guides instead of giving 
them a prior training, as is commonly done in online collaborative learning studies or peer tutoring in face-to- 
face classrooms. The results showed that five of seven CK tutors and two of seven TS tutors and applied the 
respective intervention. Our findings confirm the findings of peer tutoring studies: without training, peer 
tutors seldom demonstrate certain skills to fulfill their role tasks (Nath and Ross 2001). Our observations 
during the experiment might explain why TS tutors did not use tutoring skills: i) tutors might not have 
sufficient time to read all of the textual instructions, ii) tutors might not realize the relevance of using the 
general instructions with answering the essay question, and iii) students might have developed certain 
internal scripts of dealing with such essay questions comparing facts and thus they did not need to use our 
intervention. 

To conclude, for the full study we need to oblige tutors to apply the support of competences as we 
expected, in particular for tutors to apply tutoring skills. In addition, we should implement the full study with 
a more complex task that requires higher-order cognitive skills such as analysis, evaluation and synthesis, to 
make sure that learners benefit from the kind of knowledge sharing with others that triggers extra cognitive 
processes. 
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